Active Inference for Integrated State-Estimation, Control, and Learning

This work presents an approach for control, state-estimation and learning model (hyper)parameters for robotic manipulators. It is based on the active inference framework, prominent in computational neuroscience as a theory of the brain, where behaviour arises from minimizing variational free-energy. The robotic manipulator shows adaptive and robust behaviour compared to state-of-the-art methods. Additionally, we show the exact relationship to classic methods such as PID control. Finally, we show that by learning a temporal parameter and model variances, our approach can deal with unmodelled dynamics, damps oscillations, and is robust against disturbances and poor initial parameters. The approach is validated on the `Franka Emika Panda' 7 DoF manipulator.Comment: 7 pages, 6 figures, accepted for presentation at the International Conference on Robotics and Automation (ICRA) 202

Multiscale modeling of the methanol synthesis: from surface reaction kinetics to techno-economic analysis

Eine der größten Herausforderungen des zukünftigen Energiesystems ist die effiziente und dezentrale Produktion von chemischen Energieträgern mit geringem CO2-Fußabdruck. In diesem Sinne könnte die Umwandlung von erneuerbarem H2/CO/CO2 in Methanol ein wichtiger Zwischenschritt sein, da Methanol ein geeignetes Energiespeichermedium und ein Edukt für eine Vielzahl von Mehrwertchemikalien und flüssigen Kraftstoffen ist. Eine genaue mathematische Beschreibung der Reaktionskinetik ist die Grundlage für die Prozessoptimierung und den Entwurf neuer Anlagen und trägt zusätzlich zur Entwicklung aktiverer katalytischer Systeme bei. Das Hauptziel dieser Arbeit war es daher, das mechanistische Verständnis der Methanolsynthese an Katalysatoren auf Cu/Zn-Basis unter Berücksichtigung des Zusammenspiels zwischen dem Katalysator und den verwendeten Prozessparametern zu verbessern. Um dies zu erreichen, wurden drei Zwischenziele vorgeschlagen und umgesetzt. Das erste Zwischenziel war die Entwicklung und experimentelle Validierung eines detaillierten mikrokinetischen Modells der Methanolsynthese an Cu/Zn-basierten Katalysatoren auf der Grundlage von ab initio Dichtefunktionaltheorie (DFT) Berechnungen aus der Literatur. Dabei wurden die CO-Hydrierung, die CO2-Hydrierung und die Wassergas-Shift-Reaktion berücksichtigt. Auch die Cu/Zn-Synergie in der Katalysatormatrix wird in Betracht gezogen. Mit dem validierten Modell (mit eigenen Experimenten und Literaturdaten) wurden Erkenntnisse über die bevorzugten Reaktionswege (mittels Reaktionsflussanalyse) und die geschwindigkeitsbestimmenden Schritte (mittels Sensitivitätsanalyse) gewonnen. Das zweite Zwischenziel war die Entwicklung und experimentelle Validierung eines formalen kinetischen Modells, das aus dem mikrokinetischen Modell abgeleitet wurde, um den Rechenaufwand für die Durchführung von Simulationen erheblich zu verringern. Verschiedene Ansätze führten zu drei kinetischen Modellen, von denen das Modell-6p das beste war (6p → 6 angepasste Parameter). Bei diesem Ansatz wurden wichtige Erkenntnisse aus dem mikrokinetischen Modell verwendet, wie der bevorzugte Reaktionsmechanismus, die geschwindigkeitsbestimmenden Schritte und die vorherrschenden adsorbierten Zwischenprodukte. Die direkte CO-Hydrierung wird vernachlässigt und sechs zusammengefasste Parameter wurden an die experimentellen Daten angepasst. Das daraus resultierende Modell-6p eignet sich für modellbasierte Anwendungen, einschließlich Scale-up von Prozessen, Prozessoptimierung und detaillierte Reaktorsimulationen mit Computational Fluid Dynamics (CFD). Das dritte Zwischenziel war die Anwendung des vorgeschlagenen formalen kinetischen Modells in einer detaillierten Simulation einer Methanolanlage aus erneuerbaren H2/CO2, einschließlich Wärmeintegration und Optimierung der Prozessparameter zur Minimierung des Reaktantenverbrauchs. Das Potenzial der Einbeziehung von Zwischenkondensationsschritten zur Verbesserung des Gesamtprozesses wurde untersucht und mittels Prozess- und techno-ökonomischer Analysen mit einem konventionellen Ansatz verglichen. In dieser Arbeit wird ein schrittweiser Prozess von der detaillierten Oberflächenkinetik bis zur angewandten Reaktionstechnik vorgestellt, der Theorie und Anwendung auf systematische Weise miteinander verbindet. Daher trägt diese Arbeit nicht nur zum Verständnis der Kinetik der Methanolsynthese bei, sondern bietet auch umfassende Unterstützung für analoge künftige Projekte (z. B. die Produktion höherer Kohlenwasserstoffe)

Gamificación como estrategia didáctica: Aplicación en la formación del profesor

This research project focuses on the use of gamification as a teaching strategy, more precisely in the teachers' training during the use of educational software (ES). This will be analyzed through our case-study, the mathematical software GGBook. The starting point of our research is the assumption that the use of gamification in an educational tool of teacher training makes the learning situation more effective. Indeed, mechanisms of gamification like motivation increase interest. We correlate those mechanisms with the Theory of Kolb of the learning styles. These latter exposes four different ways individuals engage with a learning situation depending on their knowledge construction processEste proyecto de investigación se centra en el uso de la gamificación como estrategia de enseñanza, más concretamente en la formación de profesores durante el uso de softwares educativos (SE). Esto será analizado a través de nuestro estudio de caso acerca del software de educación matemática GGBook. El punto de partida de nuestra investigación es la hypothesis de que el uso de la gamificación en una herramienta educativa de formación de profesores hace que la situación de aprendizaje sea más efectiva. De hecho, los mecanismos de gamificación, como la motivación, aumentan el interés. Relacionamos estos mecanismos con la teoría de Kolb de los estilos de aprendizaj

One Risk to Rule Them All: Addressing Distributional Shift in Offline Reinforcement Learning via Risk-Aversion

Offline reinforcement learning (RL) is suitable for safety-critical domains where online exploration is not feasible. In such domains, decision-making should take into consideration the risk of catastrophic outcomes. In other words, decision-making should be risk-averse. An additional challenge of offline RL is avoiding distributional shift, i.e. ensuring that state-action pairs visited by the policy remain near those in the dataset. Previous works on risk in offline RL combine offline RL techniques (to avoid distributional shift), with risk-sensitive RL algorithms (to achieve risk-aversion). In this work, we propose risk-aversion as a mechanism to jointly address both of these issues. We propose a model-based approach, and use an ensemble of models to estimate epistemic uncertainty, in addition to aleatoric uncertainty. We train a policy that is risk-averse, and avoids high uncertainty actions. Risk-aversion to epistemic uncertainty prevents distributional shift, as areas not covered by the dataset have high epistemic uncertainty. Risk-aversion to aleatoric uncertainty discourages actions that are inherently risky due to environment stochasticity. Thus, by only introducing risk-aversion, we avoid distributional shift in addition to achieving risk-aversion to aleatoric risk. Our algorithm, 1R2R, achieves strong performance on deterministic benchmarks, and outperforms existing approaches for risk-sensitive objectives in stochastic domains

Convex Hull Monte-Carlo Tree Search

This work investigates Monte-Carlo planning for agents in stochastic environments, with multiple objectives. We propose the Convex Hull Monte-Carlo Tree-Search (CHMCTS) framework, which builds upon Trial Based Heuristic Tree Search and Convex Hull Value Iteration (CHVI), as a solution to multi-objective planning in large environments. Moreover, we consider how to pose the problem of approximating multiobjective planning solutions as a contextual multi-armed bandits problem, giving a principled motivation for how to select actions from the view of contextual regret. This leads us to the use of Contextual Zooming for action selection, yielding Zooming CHMCTS. We evaluate our algorithm using the Generalised Deep Sea Treasure environment, demonstrating that Zooming CHMCTS can achieve a sublinear contextual regret and scales better than CHVI on a given computational budget.Comment: Camera-ready version of paper accepted to ICAPS 2020, along with relevant appendice

Formal Modelling for Multi-Robot Systems Under Uncertainty

Purpose of Review: To effectively synthesise and analyse multi-robot behaviour, we require formal task-level models which accurately capture multi-robot execution. In this paper, we review modelling formalisms for multi-robot systems under uncertainty, and discuss how they can be used for planning, reinforcement learning, model checking, and simulation. Recent Findings: Recent work has investigated models which more accurately capture multi-robot execution by considering different forms of uncertainty, such as temporal uncertainty and partial observability, and modelling the effects of robot interactions on action execution. Other strands of work have presented approaches for reducing the size of multi-robot models to admit more efficient solution methods. This can be achieved by decoupling the robots under independence assumptions, or reasoning over higher level macro actions. Summary: Existing multi-robot models demonstrate a trade off between accurately capturing robot dependencies and uncertainty, and being small enough to tractably solve real world problems. Therefore, future research should exploit realistic assumptions over multi-robot behaviour to develop smaller models which retain accurate representations of uncertainty and robot interactions; and exploit the structure of multi-robot problems, such as factored state spaces, to develop scalable solution methods.Comment: 23 pages, 0 figures, 2 tables. Current Robotics Reports (2023). This version of the article has been accepted for publication, after peer review (when applicable) but is not the Version of Record and does not reflect post-acceptance improvements, or any corrections. The Version of Record is available online at: https://dx.doi.org/10.1007/s43154-023-00104-

Right Place, Right Time:Proactive Multi-Robot Task Allocation Under Spatiotemporal Uncertainty

For many multi-robot problems, tasks are announced during execution, where task announcement times and locations are uncertain. To synthesise multi-robot behaviour that is robust to early announcements and unexpected delays, multi-robot task allocation methods must explicitly model the stochastic processes that govern task announcement. In this paper, we model task announcement using continuous-time Markov chains which predict when and where tasks will be announced. We then present a task allocation framework which uses the continuous-time Markov chains to allocate tasks proactively, such that robots are near or at the task location upon its announcement. Our method seeks to minimise the expected total waiting duration for each task, i.e. the duration between task announcement and a robot beginning to service the task. Our framework can be applied to any multi-robot task allocation problem where robots complete spatiotemporal tasks which are announced stochastically. We demonstrate the efficacy of our approach in simulation, where we outperform baselines which do not allocate tasks proactively, or do not fully exploit our task announcement models

Origens e consolidação da ideia de justiça social

Este artigo pretende investigar as origens e o desenvolvimento histórico da ideia de justiça social, desde os usos iniciais da expressão, na primeira metade do século XIX, até sua consolidação definitiva em meados do século XX. O objetivo é esclarecer uma noção que, embora de uso frequente nos discursos contemporâneos, não é definida rigorosamente pelos que a empregam, gerando desavenças teóricas profundas. Três passos foram fundamentais para a afirmação da ideia: um redimensionamento do âmbito da virtude da justiça em contraposição à caridade, a elaboração de uma concepção mais abrangente da liberdade pessoal e o surgimento de uma nova categoria de direitos humanos, os direitos sociais

Jusnaturalismo e direitos humanos

Este artigo pretende mostrar que o surgimento dos direitos humanos está intimamente ligado às teorias jusnaturalistas modernas e, portanto, às noções de sujeito de direito e natureza humana